A Knowledge Integration Framework for 3D Shape Reconstruction

The modern emergence of automation in many industries has given impetus to extensive research into mobile robotics. Novel perception technologies now enable cars to drive autonomously, tractors to till a field automatically and underwater robots to construct pipelines. An essential requirement to facilitate both perception and autonomous navigation is the analysis of the 3D environment using sensors like laser scanners or stereo cameras. 3D sensors generate a very large number of 3D data points in sampling object shapes within an environment, but crucially do not provide any intrinsic information about the environment in which the robots operate with. This means unstructured 3D samples must be processed by application-specific models to enable a robot, for instance, to detect and identify objects and infer the scene geometry for path-planning more efficiently than by using raw 3D data. This thesis specifically focuses on the fundamental task of 3D shape reconstruction and modelling by presenting a new knowledge integration framework for unstructured 3D samples. The novelty lies in the representation of surfaces by algebraic functions with limited support, which enables the extraction of smooth consistent shapes from noisy samples with a heterogeneous density. Moreover, many surfaces in urban environments can reasonably be assumed to be planar, and the framework exploits this knowledge to enable effective noise suppression without loss of detail. This is achieved by using a convex optimization technique which has linear computational complexity. Thus is much more efficient than existing solutions. The new framework has been validated by critical experimental analysis and evaluation and has been shown to increase the accuracy of the reconstructed shape significantly compared to state-of-the-art methods. Applying this new knowledge integration framework means that less accurate, low-cost 3D sensors can be employed without sacrificing the high demands that 3D perception must achieve. This links well into the area of robotic inspection, as for example regarding small drones that use inaccurate and lightweight image sensors

Implicit scene modelling from imprecise point clouds

In applying optical methods for automated 3D indoor modelling, the 3D reconstruction of objects and surfaces is very sensitive to both lighting conditions and the observed surface properties, which ultimately compromise the utility of the acquired 3D point clouds. This paper presents a robust scene reconstruction method which is predicated upon the observation that most objects contain only a small set of primitives. The approach combines sparse approximation techniques from the compressive sensing domain with surface rendering approaches from computer graphics. The amalgamation of these techniques allows a scene to be represented by a small set of geometric primitives and to generate perceptually appealing results. The resulting scene surface models are defined as implicit functions and may be processed using conventional rendering algorithms such as marching cubes, to deliver polygonal models of arbitrary resolution. It will also be shown that 3D point clouds with outliers, strong noise and varying sampling density can be reliably processed without manual intervention

Visual recognition of bridges by using stereo cameras on trains

Recognition of either patterns or objects in mobile systems continues to be in the focus of intensive research, with many applications being enhanced by integrating environment related information. This paper presents a practical technique for detecting and recognizing bridges from a train using a stereo camera which provides depth and grayscale images. The algorithm has been applied to a train system, where object detection combined with a given map of an area is used to improve localization. The approach is based on the detection of primitive features including edges and corners in the depth image. The pairwise spatial relations between the features are then modeled by a graph, so the classification and detection can be performed by a probabilistic Markov Random Field framework. The algorithm has been tested on the real-life datasets of the Rail Collision Avoidance System (RCAS) project. The presented results prove the applicability of the framework for detection of objects by exploiting geometrical appearance constraints

Large Scale 3D Modelling via Sparse Volumes

Spatial 3D reconstruction received enormous interest in the last years. However, the goal to store, to process and to visualize the acquired data is still very challenging. Discrete voxel based representation techniques became state of the art in todays research approaches. These allow summary of redundant measurements and fast coordinate based access to the data leading to efficient volume computations. Unfortunately, representing the 3D space with a dense voxel grid requires huge amount of storage. Representing a volume of 100 x 100 x 100m3 with resolution of 1cm with a dense grid of 32-bit floating point values, results in a 3:8 TB storage requirement. This motivated many state of the art approaches to apply octrees to build sparse 3D volumes, where only the occupied voxels are stored. This however, increases the data access complexity from O(1) to O(d) with d as the depth of the octree, growing logarithmically when the volume or the resolution of the model is increasing. In this work we propose to combine octrees with hash tables leading to sparse voxel representation well suited for efficient storage and fast data access common in 3D modelling computations. The hash table is used to access grid cells, which further contain an octree in itself. Since the internal octrees are constructed of much smaller depth e.g. di = 1, this dramatically decreases the access time complexity to O(di). For a standard octree with depth d = 16 , this leads to a speed-up of factor 16. An additional advantage of the hash table approach is that the volume size is not limited and is suited for modelling huge environments

TVL1 Shape Approximation from Scattered 3D Data

With the emergence in 3D sensors such as laser scanners and 3D reconstruction from cameras, large 3D point clouds can now be sampled from physical objects within a scene. The raw 3D samples delivered by these sensors however, contain only a limite d degree of information about the environment the objects exist in, which means that further geometrical high-level modelling is essential. In addition, issues like sparse data measurements, noise, missing samples due to occlusion, and the inherently huge datasets involved in such representations makes this task extremely challenging. This paper addresses these issues by presenting a new 3D shape modelling framework for samples acquired from 3D sensor. Motivated by the success of nonlinear kernel-based approximation techniques in the statistics domain, existing methods using radial basis functions are applied to 3D object shape approximation. The task is framed as an optimization problem and is extended using non-smooth L1 total variation regularization. Appropriate convex energy functionals are constructed and solved by applying the Alternating Direction Method of Multipliers approach, which is then extended using Gauss-Seidel iterations. This significantly lowers the computational complexity involved in generating 3D shape from 3D samples, while both numerical and qualitative analysis confirms the superior shape modelling performance of this new framework compared with existing 3D shape reconstruction techniques

Sparse Volumes for Large Scale 3D Modelling

Modern emergence of automation in the industry and everyday life is leveraged by extensive research in mobile robotics. Novel 3D sensors such as laser scanners or cameras enable cars to drive autonomously, UAVs to observe critical environments, or an underwater robot to construct pipelines. However, 3D sensor samples do not provide the intrinsic information a robot needs to operate on. Voxel based shape modelling has been identified as a fruitful solution. However, its application is limited to small areas since processing and visualization of large environments is very challenging. Dense voxel grids allow fast data access but suffer from a large memory overhead. Modelling an area of 100x100x100m with a resolution of 1cm would result in a 3.7TB memory requirement. Motivated by this, sparse voxel octrees (SVO) [4] have been proposed. These however, increase the data access complexity fro

Integrated Positioning System (IPS) Vision Aided Navigation Technology

Integrated Positioning System (IPS) Vision Aided Navigation Technolog

Surface Reconstruction from Imprecise 3D Points

In applying optical methods for automated 3D indoor modelling, the 3D reconstruction of objects and surfaces is very sensitive to both lighting conditions and the observed surface properties which ultimately compromises the utility of the acquired 3D point clouds. This works presents a statistical method for surface reconstruction applying robust statistics from the compressive sensing discipline. It is shown that 3D point clouds with outliers, strong noise and varying sampling density can be processed by the presented method without manual interaction. The resulting surface models are stored in implicit functional form and may be processed by common rendering algorithms such as marching cubes to deliver polygonal models of arbitrary resolution

Data Challenges with 3D Computer Vision

A wide range of end user and industrial applications rely on accurate 3D scene representation. Automated 3D modelling from optical sensors such as LIDAR scanners, stereo or RGBD cameras became inevitable since this enables to replace the time consuming manual 3D modelling process. However, optical sensors deliver extremely high amount of data which is required to be processed to information. An 8 bit stereo system with 9MPx colour images delivers 4GB of data per second. Being able to make sense out of this enormous flow of data is a huge and ongoing research and development task. This talk will give an overview of the challenges which naturally involve efficient search, storage and scalable algorithms for information extraction from 3D data. Also an overview of practical implications on 3D reconstruction of cities, autonomous driving in public and industrial facilities, or inspection and monitoring as part of security strategies will be presented

Infinite 3D Modelling Volumes

Modern research in mobile robotics proposes to combine localization and perception in order to recognize previously visited locations and thus to improve localization as well as the object recognition processes recursively. A crucial issue is to perform updates of the scene geometry when novel observations become available. The reason is that a practical application often requires a system to model large 3D environments at high resolution which exceeds the storage of the local memory. The underlying work presents an optimized volume data structure for infinite 3D environments which facilitates i) successive world model updates without the need to recompute the full dataset, ii) very fast in-memory data access scheme enabling the integration of high resolution 3D sensors in real-time, iii) efficient level-of-detail for visualization and coarse geometry updates. The technique is finally demonstrated on real world application scenarios which underpin the feasibility of the research outcomes